Enhancing the Accuracy of an Image Classification Model Using Cross-Modality Transfer Learning

Applying deep learning (DL) algorithms for image classification tasks becomes more challenging with insufficient training data. Transfer (TL) has been proposed to address these problems. In theory, TL requires only a small amount of knowledge be transferred the target task, but traditional transfer often presence same or similar features in source and domains. Cross-modality (CMTL) solves this problem by domain completely different from domain, using large data, which helps model learn features. Most existing research on CMTL focused image-to-image transfer. paper, is formulated text domain. Our study started two separately pre-trained models domains obtain network structure. The was via new hybrid (combining BERT BEiT models). Next, GridSearchCV 5-fold cross-validation were used identify most suitable combination hyperparameters (batch size rate) optimizers (SGDM ADAM) our model. To evaluate their impact, 48 two-tuple well-known used. performance evaluation metrics validation accuracy, F1-score, precision, recall. ablation confirms that enhanced accuracy 12.8% compared original addition, results show can significantly impact performance.